Draft version February 27, 2012 

Preprint typeset using I^'T^]X style emulatoapj v. 2/16/10 



AN EXPONENTIAL DECLINE AT THE BRIGHT END OF THE Z = 6 GALAXY LUMINOSITY FUNCTION 

Chris J. Willott\ Ross J. McLure^, Pascale Hibon^, Richard Bielby'', Henry J. McCracken"', Jean-Paul 
Kneib'', Olivier Ilbert'', David G. Bonfield'^, Victoria A. Bruce^, and Matt J. Jarvis^, 

Draft version February 27, 2012 



O 
(N 

D 

m 



o 

6 

in: 



> 

o 

m 

O 



ABSTRACT 

We present the results of a search for the most luminous star-forming galaxies at redshifts z « 6 
based on CFHT Legacy Survey data. We identify a sample of 40 Lyman break galaxies brighter than 
magnitude z' = 25.3 across an area of almost 4 square degrees. Sensitive spectroscopic observations 
of seven galaxies provide redshifts for five, of which only two have moderate to strong Lya emission 
lines. All five have clear continuum breaks in their spectra. Approximately half of the Lyman break 
galaxies are spatially resolved in 0.7 arcsec seeing images, indicating larger sizes than lower luminosity 
galaxies discovered with the Hubble Space Telescope, possibly due to on-going mergers. The stacked 
optical and infrared photometry is consistent with a galaxy model with stellar mass ^ 10^" Mq. 
There is strong evidence for substantial dust reddening with a best-fit Ay = 0.75 and Ay > 0.48 
at 2(7 confidence, in contrast to the typical dust-free galaxies of lower luminosity at this epoch. The 
spatial extent and spectral energy distribution suggests that the most luminous z w 6 galaxies are 
undergoing merger-induced starbursts. The luminosity function of z = 5.9 star-forming galaxies is 
derived. This agrees well with previous work and shows strong evidence for an exponential decline at 
the bright end, indicating that the feedback processes which govern the shape of the bright end are 
occurring effectively at this epoch. 

Subject headings: cosmology: observations — galaxies: evolution — galaxies: high-redshift 



1. INTRODUCTION 

The light from distant galaxies brings evidence of the 
conditions and physical processes at play in the early 
Universe. By studying the changes in galaxy properties 
over cosmic time we obtain a deeper understanding of 
how our Universe evolved. At 1 billion years after the 
Big Bang, galaxies were typically smaller, less luminous 
and less dusty than today (Bouwens et al. 2006). Such 
observations need to be explained by cosmological sim- 
ulations which account for the hierarchical merging of 
dark matter halos and the gas accretion and cooling in- 
side them necessary to form stars (e.g. Finlator et al. 
2011). 

One of the key measurements of the evolving Universe 
is the galaxy luminosity function. This function is related 
to the star formation rate occurring at an epoch and how 
the star formation is distributed across the galaxy popu- 
lation. Comparison of the observed luminosity function 
with that predicted by models constrains the important 
physical processes occurring. For example, the faint end 
slope of the galaxy luminosity function is flatter than 
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the dark matter halo mass function, which could be ex- 
plained by feedback from supernovae winds (Cole 1991) 
or from photoevaporation and heating during reioniza- 
tion (Barkana & Loeb 2001). At the bright end, the 
galaxy luminosity declines much more sharply than the 
halo mass function. This is usually ascribed to AGN 
feedback and inefficient gas cooling in high mass halos 
(e.g. Benson et al. 2003). 

Surveys for UV-continuum-selected galaxies at z > 6 
have been largely focused on Hubble Space Telescope due 
to the small sizes of the galaxies and low background 
in space. Hundreds of such galaxies have been identi- 
fied in deep ACS imaging of the GOODS, Hubble Uhra 
Deep (HUDF) and parallel fields (Dickinson et al. 2004; 
Bunker et al. 2004; Yan & Windhorst 2004; Bouwens 
et al. 2006). The extremely deep imaging over the very 
small area of the HUDF identifies galaxies with abso- 
lute magnitude at 1350 A, M1350 — —18, corresponding 
to a star formation rate of only 1 Mq yr~^. These Hub- 
ble Space Telescope surveys have led to a good descrip- 
tion of the z = 6 luminosity function over the range 
-21 < M1350 < -18 (Bouwens et al. 2007, 2008; Su 
et al. 2011). The luminosity function can be fit by 
a Schechter function with characteristic break luminos- 
ity of M = —20.2. However, the few sources detected 
brighter than the break leave the exact nature of the 
high-luminosity decline uncertain. 

Several surveys have been carried out from the ground 
over wider areas to find the rarer, more luminous z « 6 
galaxies. McLure et al. (2006, 2009) and Curtis-Lake et 
al. (2011) report on 5 < z < 6.5 galaxies found within 
the 0.8 square degree Subaru/ XMM-Newton Deep Sur- 
vey (optical) and UKIDSS Ultra Deep Survey (near-IR). 
This SXDS/UDS work provides tighter constraints on 
the luminosity function at —22 < M1350 < —21 than 
from GOODS (McLure et al. 2009). Shimasaku et al. 
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(2005), Nagao et al. (2007) and Jiang et al. (2011) 
have discovered many z w 6 galaxies in the 0.25 square 
degree Subaru Deep Field (SDF) which provides good 
luminosity function constraints for —21 < M1350 < —20. 
Together these works show that the galaxy luminos- 
ity function at z = 6 can be fit by a Schechter func- 
tion, the same parameterization that successfully fits at 
lower redshift. The surveys discussed above only contain 
enough volume to discover galaxies with a space density 
of ^ 10~^ Mpc~^. They show there is a steepening of 
the luminosity function, but do not confirm whether the 
cut-off is exponential or less steep than this. 

At even higher redshifts (z ^ 7 — 8) there have been 
mixed results from ground-based surveys to determine 
the bright end of the luminosity function. Castellano et 
al. (2010) found a decrease in the space density from 
z = 6 to z = 6.8 of a factor of 3.5. Capak et al. (2011) 
found three UV-bright z ^ 7 galaxy candidates in the 
2 square degree COSMOS field. Although there is some 
evidence that these galaxies are low-z interlopers, if they 
are truly at z w 7 then the galaxy luminosity function 
does not decline precipitously as a Schechter function, 
but rather as a power-law. Luminous z « 8 LBG candi- 
dates have recently been reported by Yan et al. (2011), 
also suggesting a bright end decline less steep than a 
Schechter function. Although both the above studies 
could be affected by low-z contamination, it is important 
to determine how well the z = 6 luminosity function is 
fit by a Schechter function, because there is little cosmic 
time available between z — 6 and z = 7 for significant 
evolution of the shape of the luminosity function. Capak 
et al. noted that their result could be explained if AGN 
feedback is less effective at early times due to the time 
required for supermassive black holes to build up their 
mass via Eddington-limited accretion and mergers. 

The Deep component of the Canada-France-Hawaii 
Telescope Legacy Survey (CFHTL^ provides the deep- 
est optical data covering several square degrees and the 
largest area survey capable of finding z w 6 galaxies 
based on their rest-frame UV continuum. At a total of 
nearly 4 square degrees the volume probed is approx- 
imately 40 times that of the GOODS survey and five 
times the previous largest area studied, the SXDS/UDS. 
We present the results of a search for z « 6 galaxies in 
the CFHTLS. Section[2]describes the optical and near-IR 
data used and how the galaxies were selected. Section [3] 
gives details of spectroscopic foUowup of a subset of the 
galaxies. Section U] considers the physical sizes of these 
galaxies. In Section [5] we stack together the optical and 
IR images for the galaxy sample to demonstrate their 
high-z nature and determine the typical galaxy SED. Sec- 
tion [S] presents the resulting luminosity function and our 
conclusions are drawn in Section [71 

All optical and near-IR magnitudes in this paper are 
on the AB system. Cosmological parameters of Ho = 
70 km s~^ Mpc~-^, = 0.30 and = 0.7 are assumed 
throughout. These parameters have been adopted to ease 
comparison with previous work, even though they are 
slightly different from the best fit in Jarosik et al. (2011). 

2. IMAGING AND SAMPLE SELECTION 

^ http://www.cflit.hawaii.odu/SciGnce/CFHTLS 



2.1. Imaging observations 

The imaging data used to select high-redshift galax- 
ies come primarily from the 3.6 m Canada- France-Hawaii 
Telescope. Optical observations with MegaCam in the 
u * g'r'i'z' filters are from CFHTLS Deep which covered 
four « 1 square degree fields with typical total integra- 
tion time of 75 ks in u*, 85 ks in g', 145 ks in r', 230 ks in 
i' and 175 ks in z' . The seeing in the final stacks at i' and 
z' range from 0.66 to 0.76 arcsec. The data used here are 
from the 6th data release, T0006, which contains all the 
data acquired over the five years of the projeclH. 

These optical data are complemented by near-IR data 
from the WIRCam Deep Survey (WIRDS; Bielby et al. 
2012). WIRDS used the WIRCam near-IR imager at 
the CFHT to cover 2.4 square degrees of the CFHTLS 
Deep reaching typical 50% completeness depth of AB 
magnitude 24.5 in the JHKg filters. A few objects not 
in the regions covered by WIRDS had J band photom- 
etry obtained from the Gemini-North Telescope using 
GNIRS and from the ESO New Technology Telescope us- 
ing SOFI. More recently, near-IR data for some of these 
regions has become available from the ESO VISTA tele- 
scope. The Dl and D2 fields are fully covered by the 
first public data releases of the VIDEO survey ( Jarvis et 
al. in prep.) and UltraVISTA survey (McCracken et al. 
in prep.), respectively. These data reach about a magni- 
tude deeper than WIRDS, so all sources in Dl and D2 
have YJHK photometry in this paper from the VISTA 
surveys instead of the original WIRDS data. Note that 
the VIDEO data used has been corrected for the non- 
optimal sky subtraction detailed in the release notes. 
Small parts of the D2 and D3 field are covered by the 
Hubble Space Telescope Multi-Cycle Treasury program 
CANDELS (Grogin et al. 2011; Koekemoer et al. 2011). 

Photometry was carried out in dual-image mode us- 
ing the Sextractor source extractor software (Bertin & 
Arnouts 1996). The z' band was used as the detec- 
tion band because it provides the highest signal-to-noise 
(S/N) for z « 6 galaxies. 2 arcsec diameter photomet- 
ric apertures were used. Aperture magnitudes were cor- 
rected to total magnitudes assuming that the objects are 
spatially unresolved. This gives a lower limit on the fiux 
for spatially extended objects. 

2.2. Sample Selection 

There are two methods for selecting high-redshift 
galaxies from optical/IR broad-band imaging, Lyman 
break and photometric redshifts. The Lyman break tech- 
nique adopts hard color cuts in one or more colors pos- 
sibly including non-detections in certain filters, whereas 
photometric redshifts use all available filters to derive a 
redshift probability distribution. Photometric redshifts 
are most suitable when the targets will be detected in 
many filters, however they reduce to a Lyman break- 
type selection if only a few filters are deep enough to 
constrain the relevant objects. In this work we use the 
Lyman break technique for homogeneous selection be- 
cause of the variable near-IR data quality in the different 
Deep fields. 

We set a magnitude limit of z' < 25.3 to ensure that 
the objects are not too faint for our data. This mini- 

http:/ /terapix.iap.fr/rubriquo.php?id_mbrique=259 
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mizes the problem of photometric scatter of objects into 
our sample with true colors different from the selection 
criteria. The luminosity function has been well studied 
by others at magnitudes fainter than this limit. The 
primary selection criterion is color i' — z' > 2 which cor- 
responds to the break across the Lya line. This criterion 
is somewhat stricter than other studies (e.g. Bouwens et 
al. 2006; Jiang et al. 2011) but ensures that contamina- 
tion from low redshift galaxies and brown dwarfs is kept 
to a minimum. 

Most Lyman break surveys adopt two colour criteria 
and therefore define a box in two-dimensional color-color 
space. We adopt a similar, but slightly different, method. 
Instead of a hard cut-off in z' — J color we consider all 
the properties of every source in the i' — z' > 2 selec- 
tion region and determine whether it is most likely a 
high-redshift galaxy or something else. This is important 
here because the CFHTLS Deep Fields contain varying 
amounts of extra multi-wavelength data which can be 
used as an additional constraint. 

The initial automated search routine revealed 136 pos- 
sible candidates in the ~ 4 square degrees CFHTLS 
Deep. The images were inspected by eye and sometimes 
more detailed manual photometry for objects in locations 
with varying background was performed. Non-detections 
in the CFHTLS u*g'r' filters were also required for good 
LEG candidates. The most common problems leading to 
rejection from the sample were due to structured back- 
ground near bright stars or close to the edges of the fields 
where the effective exposure time is lower. A total of 69 
candidates were removed from the list in this process. 
This left 67 true astronomical sources with i' — z' > 2. 

For each of these, the available data was studied to 
determine its nature. The primary filter for this pro- 
cess is J because brown dwarfs are known to have much 
higher z' — J colors than high-redshift galaxies. 7 of 
the 67 sources do not have J band coverage. The next 
most important is Ks band because reddened galaxies at 
z « 1 would be expected to be bright at Ks. In addi- 
tion, three of the fields have at least partial Spitzer Space 
Telescope IRAC coverage available from SWIRE (Lons- 
dale et al. 2003), S-COSMOS (Sanders et al. 2007) and 
AEGIS (Barmby et al. 2008). The COSMOS field also 
has deep Subaru imaging with broad-band depth similar 
to CFHTLS, additional medium-band filters (Taniguchi 
et al. 2007) and HST F814W imaging (Scoville et al. 
2007) which is useful for compact sources. 

The results of this process showed that 40 of the 67 
candidates were most likely z ~ 6 Lyman break galaxies 
and the remaining 27 are most likely to be brown dwarfs. 

2.3. Evidence for z w 6 galaxies 

Three-colour {r'i'z') images for the the 40 candidates 
classified as high-redshift galaxies are shown in Figure [TJ 
Photometric measurements in the i' z' J filters are given 
in Table [TJ In the case that an object is not detected, 
a magnitude limit is given. These magnitude limits are 
determined from the variance of photometric aperture 
fluxes at random locations close to the object. At i' 
band, many sources appear to be marginally detected. 
Therefore i' magnitudes are quoted down to magnitude 
errors of 0.5 magnitudes, rather than the strict 2a limits 
which are often brighter. This should be kept in mind if 
using i' magnitudes or i' — z' colors from Table [1] Note 
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Figure 1. Color images {r'i' z') for tiie sample of 40 2 ~ 6 Lyman 
break galaxies. Each image covers 15" X 15" and is oriented with 
north up and east to the left. The LBGs are the small red objects 
at the centers. Some are clearly spatially extended. 



that six of the 40 galaxies do not have J band observa- 
tions. The evidence in favor of these being high-redshift 
galaxies will be discussed later. The final column gives 
the measured FWHM in the z' band. Many of the galax- 
ies appear more extended than the seeing, a fact that will 
be discussed later. 

Figure [5] shows the distribution of points on the i' — z' 
vs z' — J color-color diagram for the 60 of 67 candidates 
which are covered by J band observations. Almost all 
the sources detected at J band have colors close to the 
expected loci of z > 5.7 Lyman break galaxies or brown 
dwarfs. Close to the i' — z' = 2 boundary the dwarfs 
tend to lie below the line. A similar tendency was noted 
for the Canada- France Brown Dwarf Survey (Reyle et al. 
2010) and is at least partially due to photometric scatter 
of somewhat bluer dwarfs with intrinsic i' — z' ~ 1.5. 
Note that a similar effect will occur for the high-redshift 
galaxies. This is why we began with a relatively high 
i' — z' cut, so that even those scattered from lower i' — z' 
have intrinsic colors matching only brown dwarfs or high- 
z galaxies. The brown dwarfs in the CFHTLS/WIRDS 
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Table 1 

Position and photometry of CFHTLS 2 = 6 Lyman break galaxies 



Name 




R.A. and Dccl. (J2000.0) 


i' mag 


z' mag 




J mag 


i' -z' 


z'~J 


FWHM (arcsec) 


WMH 


1 


02: 


:24:13.79 


—04:56:41.4 


27.18 ± 0.26 


25.15 ± 0, 


,10 


> 25.00 


2.03 


< 0.15 


1.36 


WMH 


2 


02: 


:24:15.10 


—04:20:47.0 


27.29 ± 0.29 


25.14 + 0, 


.10 


24.58 ± 0.24 


2.15 


0.56 


0.92 


WMH 


3 


02: 


:24:51.14 


—04:03:29.4 


27.42 ± 0.32 


24.78 ± 0, 


.07 


24.57 + 0.24 


2.64 


0.21 


0.94 


WMH 


4 


02: 


:25:19.64 


—04:28:06.8 


27.47 ± 0.42 


25.21 ± 0, 


.13 


> 25.30 


2.26 


< -0.09 


0.88 


WMH 


5 


02: 


:26:27.03 


—04:52:38.3 


27.47 ± 0.34 


24.54 ± 0, 


.06 


24.17 + 0.17 


2.93 


0.37 


1.11 


WMH 


6 


02: 


:27:18.77 


—04:50:08.4 


26.97 ± 0.22 


24.85 ± 0, 


.08 


25.16 ± 0.39 


2.12 


-0.31 


1.94 


WMH 


7 


02: 


:27:29.03 


-04:33:04.3 


27.69 ± 0.48 


25.26 ± 0, 


.11 


24.87 + 0.31 


2.43 


0.39 


1.45 


WMH 


8 


02: 


:27:46.20 


—04:30:32.2 


27.46 ± 0.34 


25.16 ± 0, 


.10 


24.35 ± 0.20 


2.30 


0.81 


1.73 


WMH 


9 


09: 


:58:45.49 


+02:23:24.8 


27.67 ± 0.49 


25.27 + 0, 


,12 


24.84 ± 0.30 


2.40 


0.43 


1.21 


WMH 


10 


09: 


:58:59.84 


+01:59:48.8 


27.42 ± 0.37 


25.24 ± 0, 


.12 


25.16 ± 0.39 


2.18 


0.08 


0.84 


WMH 


11 


09: 


:59:44.49 


+02:09:36.7 


27.29 ± 0.33 


25.06 ± 0, 


.10 


24.90 ± 0.32 


2.23 


0.16 


1.02 


WMH 


12 


09: 


:59:52.74 


+02:25:53.2 


27.34 ± 0.41 


25.17 + 0, 


,13 


25.03 ± 0.35 


2.17 


0.13 


0.99 


WMH 


13 


09: 


:59:56.54 


+02:12:27.1 


26.78 ± 0.21 


24.78 ± 0, 


.08 


24.53 ± 0.23 


2.00 


0.25 


0.86 


WMH 


14 


10: 


: 00:19. 93 


+02:25:36.8 


26.98 ± 0.26 


24.88 ± 0, 


,09 


24.62 ± 0.25 


2.10 


0.26 


0.74 


WMH 


15 


10: 


: 00:26. 37 


+02:13:46.8 


> 27.35 


24.99 ± 0, 


,10 


25.05 ± 0.36 


> 2.36 


-0.06 


1.24 


WMH 


16 


10: 


: 00:30. 58 


+02:19:35.1 


27.56 ± 0.42 


25.27 + 0, 


.12 


24.85 ± 0.31 


2.29 


0.42 


1.11 


WMH 


17 


10: 


: 00:50. 10 


+02:13:03.3 


27.37 ± 0.38 


25.17 + 0, 


,12 


24.60 ± 0.25 


2.20 


0.57 


1.05 


WMH 


18 


10: 


: 00:59. 82 


+01:57:26.6 


> 27.27 


25.24 ± 0, 


,15 


24.63 ± 0.26 


> 2.03 


0.61 


1.10 


WMH 


19 


10: 


:01:18.99 


+02:11:11.6 


27.64 ± 0.46 


25.15 ± 0, 


.11 


> 25.20 


2.49 


< -0.05 


1.29 


WMH 


20 


10: 


:01:21.54 


+02:12:22.5 


26.83 ± 0.22 


24.65 ± 0, 


,07 


24.14 ± 0.17 


2.18 


0.51 


0.86 


WMH 


21 


10: 


:01:21.94 


+02:19:37.6 


27.20 ± 0.30 


25.12 ± 0, 


,10 


24.15 ± 0.17 


2.08 


0.97 


1.25 


WMH 


22 


10: 


:01:38.37 


+01:43:48.1 


27.43 ± 0.45 


25.28 ± 0, 


.13 


> 25.20 


2.15 


< 0.08 


0.81 


WMH 


23 


10: 


:01:38.71 


+02:28:20.0 


> 27.16 


25.05 ± 0, 


,12 


24.31 ± 0.19 


> 2.11 


0.74 


1.32 


WMH 


24 


14: 


:16:20.38 


+52:13:23.4 


> 27.07 


24.72 ± 0, 


.07 


— 


> 2.35 


— 


1.03 


WMH 


25 


14: 


:17:47.97 


+52:38:09.4 


> 27.28 


25.04 ± 0, 


.10 


> 25.19 


> 2.24 


< -0.15 


1.53 


WMH 


26 


14: 


:18:10.47 


+52:19:43.4 


27.79 ± 0.45 


25.13 ± 0, 


,10 


— 


2.66 


— 


1.24 


WMH 


27 


14: 


:18:40.04 


+52:35:08.2 


27.20 ± 0.32 


25.09 ± 0, 


.10 


> 24.61 


2.11 


< 0.48 


1.15 


WMH 


28 


14: 


:18:52.23 


+53:07:47.3 


27.31 ± 0.29 


25.23 ± 0, 


.11 


> 24.16 


2.08 


< 1.07 


1.00 


WMH 


29 


14: 


:19:20.53 


+52:52:38.7 


26.52 ± 0.14 


24.41 + 0, 


,05 


24.42 ± 0.05 


2.11 


-0.01 


1.11 


WMH 


30 


14: 


:20:40.80 


+52:52:45.3 


27.47 + 0.34 


24.72 + 0, 


.07 


> 24.75 


2.75 


< -0.03 


1.08 


WMH 


31 


14: 


:21:03.74 


+52:12:16.2 


26.90 + 0.21 


24.87 + 0, 


.08 




2.03 




1.08 


WMH 


32 


22: 


:13:42.68 


-17:56:33.2 


27.42 + 0.38 


25.01 + 0, 


,10 


> 24.44 


2.41 


< 0.57 


0.77 


WMH 


33 


22: 


:14:06.97 


-17:37:59.7 


> 27.21 


25.24 + 0, 


.13 




> 1.97 




0.73 


WMH 


34 


22: 


:14:46.63 


-17:39:22.2 


27.31 + 0.35 


24.98 + 0, 


,11 




2.33 




1.02 


WMH 


35 


22: 


:15:04.28 


-17:57:20.5 


> 27.18 


25.03 + 0, 


.12 


> 24.48 


> 2.15 


< 0.55 


1.44 


WMH 


36 


22: 


:15:48.91 


-17:35:45.7 


27.30 + 0.34 


25.27 + 0, 


.14 


> 24.85 


2.03 


< 0.42 


1.17 


WMH 


37 


22: 


:16:26.67 


-18:04:45.0 


> 27.25 


25.26 + 0, 


,13 


> 24.12 


> 1.99 


< 1.14 


0.69 


WMH 


38 


22: 


:16:38.04 


-17:37:00.9 


27.50 ± 0.44 


25.21 + 0, 


,14 


> 24.85 


2.29 


< 0.36 


0.73 


WMH 


39 


22: 


:17:08.83 


-17:38:43.9 


26.86 + 0.24 


24.61 + 0, 


.08 


24.57 + 0.26 


2.25 


0.04 


1.53 


WMH 


40 


22: 


:17:13.55 


-18:11:45.7 


27.15 + 0.34 


25.03 ± 0, 


,10 




2.12 




0.80 



Notes. All magnitudes are on the AB system, i' and z' photometry is from CFHTLS. J photometry is from VIDEO for Dl, 
UltraVISTA for D2 and WIRDS for all objects in D3 and D4 except WMH 29 which uses CANDELS data. Spatial size (FWHM) is 
measured on the CFHTLS z' images. 



survey also have well-defined loci in the z' — J vs J — 
H and J — H vs H — Ks diagrams with no significant 
outliers. 

The gray circles show colors of model galaxies at 
0.5 < z < 3 determined using the 2008 stellar population 
synthesis models of Chariot & Bruzual (priv. comm). 
Passively evolving and dusty star-forming galaxies of all 
ages are included. Extreme dust extinction with optical 
depth up to T = 10 is included, because extreme dust is 
required to get such large i' — z' colors. Note that due to 
the larger wavelength difference between z' and J than 
between i' and z' , the effect of extreme dust reddening is 
to push low-z galaxies into the brown dwarf region of the 
diagram. The low-z galaxy models closest to the loca- 
tion of the LBGs have z « 1.2 and r « 10. Such galaxies 
would be extremely rare and faint. 

21 of the 34 LBG-classified targets with J band data 
are detected at J. The other 13 have z' — J limits well 
separated from the regions covered by brown dwarfs and 
reddened 0.5 < z < 3 galaxies. We expect almost all the 
red squares in Figure [2] to be high-redshift galaxies as 
discussed below. 

As noted in Table 1, many of the LBGs appear spa- 



tially resolved at z' band. Section U] discusses this in 
much more detail and it is concluded that at least half of 
the LBGs are spatially resolved from the ground. In ad- 
dition, Hubble Space Telescope observations of a subset 
also show that they are resolved. This provides further 
evidence against contamination by brown dwarfs. 

A further line of evidence for the 40 targets being al- 
most all true high redshift galaxies comes from their typ- 
ical colors determined by median stacking. The stack- 
ing procedure and results will be described fully in Sec- 
tion [SI but the colors of i' - z' = 2.44 ± 0.14 and 
z' — J — 0.29 ± 0.06 are plotted in Figure[21 These colors 
(along with the other near- and mid-IR colors) are con- 
sistent with those expected for a typical z = 5.9 galaxy. 
They show that contamination by either brown dwarfs or 
low-z reddened galaxies is at a low level for this sample. 

A final source of contamination to consider is contami- 
nation by quasars at z w 6 which will have similar colors. 
A search for such quasars going to magnitude z' < 24.5 
found none in the CFHTLS Deep survey area (Willott et 
al. 2010), but one dX z — 6.01 with magnitude z' = 24.4 
in the SXDS/UDS (McLure et al. 2006; Willott et al. 
2009). At such faint magnitudes, the quasar luminosity 
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Figure 2. Color-color diagram for the i'—z' > 2 sources with i'z' J 
imaging. Those classified as z ^ 6 LBGs are plotted as red squares 
whereas those classified as brown dwarfs are plotted as blue star 
symbols. The colors of the stacked LBG (Section[5| are shown with 
a large green circle. The thick black dashed line labeled "Median 
model dwarf" is the expected locus for brown dwarfs based on 
measurements of closer brown dwarfs. The "Median model LBG" 
curve shows the typical colors expected for LBGs with a ran ge of 
UV continuum slope and Ly« emission, as described in Section [6.1l 
The gray circles show the possible colors for simulated galaxies at 
redshifts 0.5 < z < 3 (see text for details). 

function slope is likely to be fairly flat (Willott et al. 
2010), so the expectation would be for « 1 more quasar 
in the magnitude range 24.5 < z' < 25.3. 

2.4. Sources without J photometry 

7 of the 67 i' — z' > 2 sample do not have J photometry 
because they lie outside the WIRDS and VISTA surveys 
and their nature, high-z galaxy or brown dwarf, had to 
be ascertained using other available information. One is 
in the D4 field and a WIRCam Ks band image provided 
by Genevieve Soucail shows it to have Ks = 22. This 



red color of 



Ks — 2.8 is consistent with a L dwarf. 



The six remaining objects are retained in the list of 
likely z « 6 galaxies. WMH 24, WMH 26, WMH 31 and 
WMH 34 have measured z' band FWHM > 1 arcsec and 
therefore appear to be spatially resolved. As will be dis- 
cussed in Section m only 15% of brown dwarfs are found 
to have such high FWHM. WMH 33 has FWHM=0.73, 
so is consistent with being unresolved. However it is 
faint at z' band with z' = 25.25 and most sources close 
to the detection limit are galaxies. It is not detected in 
a WIRCam Ks band image that reaches to Ks = 22. If 



it were a L dwarf it would be expected to have Ks sa 22 
to 22.5, so this limit, although not definitive, suggests 
that it may be a galaxy. Therefore it is retained in the 
galaxy sample. WMH 40 has FWHM=0.80 arcsec and 
z' — 25.05. It is retained within the galaxy sample but we 
note it has almost equal probability to be a high-z galaxy, 
or a brown dwarf with these properties. In conclusion, at 
least four of these six objects are almost certainly high-z 
galaxies. Taking into consideration there could be one 
z « 6 quasar in our sample, the contamination of our 
z ss 6 galaxy sample is most likely < 10%. 

3. SPECTROSCOPY 

3.1. Observations 

The photometrically-selected high-z galaxies are 
spread across nearly 4 square degrees. Given that the 
typical field-of-view of multi-object spectrographs on 
large telescopes is <C 1 degree, there is little opportunity 
for a high-multiplex factor. Spectroscopic observations 
have been attempted for 7 of the 40 candidates. Prior- 
ity was given to those which are brightest at z' band to 
minimise the integration time required. All spectroscopy 
was performed using the CMOS spectrographs on the 
8.2 m Gemini Telescopes. The R400 gratings were used 
with 1 arcsec slits to give a resolution of i? = 1000. CCD 
pixels were binned by a factor of two in the spectral di- 
rection so that each binned pixel covers 1.34 A. WMH 13 
was observed in long-slit mode, WMH 29 was observed 
equally in long-slit and multi-object mode, and all other 
observations used multi-object mode. Occasionally, two 
z sa 6 galaxies could be fit on the same mask. The masks 
were filled with lower redshift Lyman break galaxies and 
photometric redshift selected galaxies (z > 3.5). All ob- 
servations used the nod-and-shuffle mode to ensure good 
subtraction of the sky background spectrum. 

The long-slit observations were performed in queue- 
mode in excellent conditions in 2009. The multi-object 
observations were performed in classical mode on the 
nights of 12 December 2010, 27 and 28 June 2011 in 
variable conditions. Details of the spectroscopic obser- 
vations and results are given in Table |3l Due to the 
faintness of these targets, the typical continuum S/N per 
pixel between 8500A and 9000A was only 0.5. There- 
fore, redshifts could only be quickly and unambiguously 
determined for the few sources which showed very strong 
Lya emission lines. However, even at S/N — 0.5 per 
pixel, the continuum can be significantly detected over 
broader wavelength intervals and redshifts determined 
from continuum breaks (Spinrad et al. 1998). 

3.2. Model fitting 

To fully utilize the spectral information, a routine was 
developed to fit the observed spectrum to model tem- 
plates. The galaxy models used are based on the ob- 
served composite z = 3 Lyman break galaxy spectrum of 
Shapley et al. (2003). Two galaxy models are used: one 
with a strong Lya emission line and one with only Lya 
absorption. The fitting routine allows a combination of 
these models to represent a range in Lya strength. IGM 
absorption is accounted for using the model of Songaila 
(2004). The galaxy spectrum is multiplied by a power- 
law to account for the variation in observed UV contin- 
uum slope of galaxies. There are a total of four free pa- 
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Table 2 

GMOS spectroscopy of z «i 6 Lyman break galaxies 



Name Integ. M1350 Zspcc QF'' VFl^,,*" HWHMLyt,'^ 

(hr) (A) (kms-i) 



WMH 5 


2.5 


-22.65 


6.068 


A 


13 ±4 


220 ± 30 


WMH 6 


2.5 


-21.75 


5.645 


C 






WMH 13 


2.5 


-22.06 


5.983 


A 


27 ±8 


210 ± 50 


WMH 15 


4.0 


-22.02 


5.943 


B 






WMH 29 


5.0 


-22.27 


5.757 


B 


4±2 


unres.? 


WMH 34 


2.3 


-21.67 


5.759 


C 






WMH 39 


2.0 


-22.06 


5.733 


B 






Serendip 


2.5 


-21.33 


5.618 


A 


31 ± 11 


240 ± 40 




6500 7000 7500 8000 8500 9000 9500 

Wavelength (A) 



Figure 3. Optical spectra of 2 6 Lyman break candidates 
(black) with the best fitting galaxy models (red). The best-fit 
redshifts and reduced are quoted. All spectra are smoothed by 
20 pixels (27 A). Note that template fitting was performed on the 
unsmoothed spectra. The noise (dominated by the sky) is plotted 
in green and offset from zero. The noise is not to scale, but included 
to show when observed spectral features may be due to noise. The 
dashed lines mark the wavelengths of the Lya transition for the 
best-fit redshifts. 

0.5. 

All four of the galaxies with Lya emission show signif- 
icant spectral breaks across the Lya line. All four have 
best fit galaxy spectra at redshifts equal to their Lya 
redshifts. This is not too surprising for those three with 
strong Lya lines, because inability to fit the line would 
increase the . However it is encouraging for WMH 29, 
which has only weak Lya, where the continuum break 
contributes most of the weight in the fit. For the four 
galaxies without Lya emission, we depend upon con- 
tinuum breaks to constrain the redshift. Some of these 
galaxies were observed in sub-optimal conditions. Those 
with the highest noise are WMH 6, WMH 13 and WMH 
34. WMH 15 shows a clear continuum break at z — 5.94 
(the apparent positive continuum at 7200 A to 7600 A is 
due to contamination). WMH 39 also shows a break at 
z = 5.73. These two are assigned QF='B' based on these 
breaks. The other two galaxies, WMH 6 and WMH 34, 
show continuum flux in the z' band and regions with no 
flux at shorter wavelength, but the exact break redshifts 
are not well-constrained, so are assigned QF='C'. These 
two spectra have the highest noise (along with WMH 13 
which has the strongest Lya line) which likely accounts 
for the lack of a clear redshift solution. The best fit red- 
shift of z = 5.645 for WMH 6 is not very likely given its 
color of i' — z' = 2.1 (see Section [O]) . 



QF is the Quality Flag of the redshift determination. See text. 
Note that QF=C redshifts are uncertain and Zgpcc merely gives 
the best fit redshift. 

^ Lya rest-frame equivalent width, quoted only if Lya is detected. 
Uncertainties in H^Lya ^.re dominated by continuum placement. 

Half-width half-maximum of the Lya line measured from the 
peak to the red side of the line. These values have been decon- 
volved for the instrumental dispersion. The measured HWHMLya 
for WMH 30 is 150 it 60, which is equal to the instrumental dis- 
persion, but with significant uncertainty due to its low equivalent 
width. 

rameters: normalization, redshift, power-law slope and 
fraction of Lya template. The fraction of Lya template 
was set to zero for those galaxies without obvious Lya 
emission in their 2D spectra to avoid over-fitting of noise 
peaks with the Lya line. 

Best-fit galaxy models are determined by fitting the 
models to the observed spectra. The uncertainty on each 
spectral pixel due to the sky noise is determined from the 
variance of blank sky pixels in each of the masks. The 
best fit is determined by the lowest value of the reduced 
X^- The spectra, along with best fit galaxy models, are 
shown in Figure O The observed spectra, models and 
sky noise spectrum have been smoothed by 20 pixels (27 
A) for display purposes. 

3.3. Results of spectroscopy 

Each target has a redshift Quality Flag in Table [3] 
which gives a measure of the confidence of the redshift 
determined. The categories follow those in Vanzella et al. 
(2009) where 'A' means unambiguous, 'B' is likely and 
'C is uncertain. Strong Lya emission was only found 
in two of the z « 6 galaxies, WMH 5 and WMH 13. 
These are assigned QF=A. In addition, weak, but still 
significant, Lya was observed in WMH 29 which is as- 
signed QF=B. Note that this Lya line aX z — 5.757 is in 
a region of very low sky noise. It is doubtful that such 
a line would have been significantly detected at slightly 
higher redshift. The apparent emission hne at 8400 A m 
WMH 15 in Figure [3] is found to be due to sky lines upon 
examination of the full resolution spectrum. 

One of the serendipitous mask-filling galaxies with pho- 
tometric redshift z > 3.5 was also found to have strong 
Lya at z — 5.618, just below the redshift range of our 
sample. It is labeled "Serendip" and is the last object 
in Tableland Figure [3l It has QF=A and is located at 
02:26:37.02 -04:55:20.4. This galaxy has i' - z' = 1.1, 
well below our selection criterion of i' — z' — 2.0. The 
reason for this much lower color is that the strong Lya 
line is still within the i' filter at this redshift. When an 
emission line of this strength shifts from the i' band to 
the z' band (at z « 5.8) it increases the i' — z' color by 
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Table [3] includes the absolute magnitudes, Afiaso of 
the galaxies. These were calculated from the z' band 
magnitudes after taking account of the observed Lya 
emission line contributions. There is uncertainty in the 
fc-corrections due to the poorly constrained continuum 
spectral slopes (see Section l6T|) . A Bayesian prior us- 
ing the evolving luminosity function of Bouwens et al. 
(2008) was used in the derivation of the fc-corrections 
to include the effect of more common, lower luminosity 
galaxies being more likely than rarer, higher luminosity 
galaxies. 

With absolute magnitudes ranging from —21.67 to 
—22.65, the spectroscopic targets are some of the most lu- 
minous LBGs known at z ~ 6. For comparison, the break 
in the z ss 6 luminosity function is at M1350 = —20.2 
(Bouwens et al. 2007). WMH5 has M1350 = -22.65, 
which is even more luminous than the lowest known lu- 
minosity z « 6 quasar, CFHQS J0216-0455 (M1450 = 
-22.21, Willott et al. 2009). WMH5 is only 2.1 arc- 
sec from the center of a galaxy in Figure [T] This is 
an inclined disk galaxy with a photometric redshift of 
z — 1.01, rest- frame S-band absolute magnitude of 
Mb — —21.5 and disk half-light radius 4kpc. Using the 
results of studies of the TuUy-Fisher and similar relations 
for z = 1 disk galaxies (Dutton et al. 2011; Miller et al. 
2011), we find the circular velocity for this galaxy should 
be ~ 160kms~^. Modeling the gravitational potential 
as an isothermal sphere, the gravitational lensing magni- 
fication of WMH 5 due to this galaxy is 1.27. Hence the 
intrinsic absolute magnitude of WMH 5 is still extremely 
luminous with M1350 ~ —22.4. 

4. GALAXY SIZES 

An important cosmological observation is that galax- 
ies are smaller at higher redshifts than similar galax- 
ies at lower redshift (Ferguson et al. 2004; Bouwens et 
al. 2004). This is expected due to the evolution in the 
virial radii of dark matter halos, but the exact nature 
of the evolution also depends upon details of gas accre- 
tion, retention and star formation efficiency. The faint 
z ^ 6 galaxies discovered in GOODS and HUDF typi- 
cally have half-light radii of « 0.1 arcsec (Bouwens et al. 

2006) and are therefore not expected to be resolved in 
typical seeing-limited, ground-based observations. Table 
[1] showed that the measured z' band FWHM of many 
CFHTLS z « 6 LBGs are larger than the seeing of w 0.7 
arcsec of the images. 

Figure |3] plots the FWHM against magnitude for the 
40 LBGs and 27 brown dwarfs. The brown dwarfs are 
included as a comparison sample, because they are ex- 
pected to be almost all unresolved. Although many 
brown dwarfs exist in binary systems (Burgasser et al. 

2007) only a small fraction would have the relevant com- 
ponent separations to appear as a single, resolved source 
in these images. The spread in observed FWHM for 
dwarfs is due to three factors: the small difference in 
quality between the four Deep fields, the variation in 
stellar FWHM from the field centers to the edges and 
photometric noise. The photometric noise is evident by 
the increased spread in FWHM for dwarfs at fainter mag- 
nitudes. It is for this reason that we use the dwarfs as the 
unresolved comparison sample to compare to the LBGs 
rather than stars at brighter magnitudes. 

Figure |4] also shows the 25%, 50% and 75% centiles 
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Figure 4. Observed size of the selected Lyman break galaxies 
(red squares) and brown dwarfs (blue stars). It is evident that the 
Lyman break galaxies tend to have larger sizes than the unresolved 
brown dwarfs. The solid line shows the median measured FWHM 
for artificial point sources add ed to the images in our completeness 
simulations (see Section 16. It . The typical measured FWHM of 
point sources increases at faint magnitudes due to noise. The 25% 
and 75% quartiles are shown with dotted lines. The distribution 
of brown dwarfs is consistent with these lines, whereas about 50% 
of the LBGs are truly spatially resolved. The spectroscopically 
confirmed LBGs (QF='A' or 'B'; Section[3ll are enclosed by a larger 
circle. Four of these five are spatially resolved. 

of the measured FWHM distribution as a function of 
magnitude for simulated point sources inserted into the 
data (see Section l6T|) . The observed distribution for the 
dwarfs agrees well with the simulation. Note that the 
three dwarfs with measured FWHM > 1 arcsec all have 
FWHM sa 0.6 in the WIRDS J band imaging which has 
higher S/N than z' band for these red sources, confirm- 
ing photometric noise as the reason for a small fraction of 
large FWHM values. It is found that 3 out of 20 (15%) 
dwarfs with magnitudes in the range of LBGs are ob- 
served to have FWHM > 1 arcsec and lie beyond the 
75% centile due to noise. In contrast, we find that 26 
out of the 40 LBGs (63%) have FWHM > 1 arcsec and 
that the same percentage lie beyond the 75% centile. As- 
suming that 15% of these LBGs are outliers due to noise, 
then there remain w 50% which are truly resolved in our 
ground-based imaging. 

Given the uncertainty in the broadening of the FWHM 
values by seeing and noise, it is not possible to reliably 
deconvolve each value to determine the intrinsic size dis- 
tribution of our LBG sample. However, a typical in- 
trinsic half-light radius of 2kpc (0.35 arcsec) is required 
to observe a galaxy with FWHM 1.1 arcsec (median for 
our LBG sample) when the FWHM of a point source 
would be 0.85 arcsec (50% centile for the simulated point 
sources with magnitude z' — 25.0). This contrasts with 
the typical half-light radius of 1 kpc for z = 26 galaxies 
in GOODS (Bouwens et al. 2006). Our finding is con- 
sistent with an extension to the brightest galaxies of the 
strong correlation between UV luminosity and linear size 
at z = 6 presented by Dow-Hygelund et al. (2007). 

One of the spectroscopically-confirmed LBGs, WMH 
29, lies within the CANDELS survey and therefore has 
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Figure 5. WFC3 F125W and F160W imaging of WMH 29 (left 
panels), the GALFIT best fit single galaxy models (center) and 
residuals after subtraction of the smooth galaxy profiles (right). 
The images on each row are at the same greyscale level. All images 
are oriented at an angle of 50° counter-clockwise to the standard 
north-up, east-left orientation. Both filters show a residual linear 
feature extending almost 1 arcsec to the lower-left of the compact 
galaxy. 

high-resolution Hubble Space Telescope ACS and WFC3 
imaging available. We have retrieved these data in order 
to get a more detailed view of the galaxy morphology 
for one of our very luminous (M1350 = —22.27) LBGs. 
This galaxy is labeled in Figure |3] and with FWHM of 
1.11 arcsec the observed size is at the median of the LBG 
sample. The WFC F125W and F160W images show a 
fairly compact central galaxy with a tail or merging com- 
panion extending almost 1 arcsec from it (Figure [5]). We 
have used GALFIT (Peng et al. 2010) to fit galaxy 
models to the main galaxy component in each filter in- 
dependently. The intrinsic half-light radii, taking into 
account the PSF (point spread function) broadening, are 
0.09 and 0.08 arcsec (equivalent to 0.54 and 0.49 kpc), 
respectively for F125W and F160W. These values are be- 
low the median for fainter galaxies in GOODS, so for this 
one example we do not find a large disk in a highly lu- 
minous galaxy. The residual extended component (right 
panel of Figure [5]) contributes 20% of the total flux. It 
is likely this component that accounts for the resolved 
nature in the ground-based imaging. Clearly higher res- 
olution imaging is important to determine what fraction 
of the galaxies with large sizes are due to multiple com- 
ponents in interacting systems. 

Six of the 15 galaxies in D2 are well-detected in the 
ACS F814W observations of the COSMOS field. They 
are all spatially resolved and half of them show multiple 
components. The F814W filter extends further redward 
than the CFHT i' filter, so such detections in F814W are 
consistent with the z w 6 LBG classification. Therefore 
we expect multiple components and interactions/mergers 
to be important for the sample of luminous star-forming 
galaxies. An analysis of the exact relationship between 
star formation rate and mergers will require a larger sam- 
ple with high resolution imaging. 

5. STACKED GALAXY PROPERTIES 

In order to check that the typical LBG properties 
match those of high-redshift galaxies and to get an idea 
of the typical galaxy spectral shape, we have stacked the 
photometric data at the positions of the z ~ 6 galax- 
ies. After registering the images, the median fiux of 
each pixel was determined since it provides the most ro- 
bust estimator that is least affected by outliers (White 
et al. 2007). Because the near-IR data from VISTA 



has an extra filter (Y) and is deeper than WIRDS, we 
just use the 23 galaxies in Dl and D2 with VISTA data. 
We have checked that the stacked optical and near-IR 
magnitudes for D3 and D4 are consistent. Deep Spitzer 
Space Telescope IRAC data is available for the 15 galax- 
ies in D2 from the S-COSMOS survey (Sanders et al. 
2007). We include these data because these wavelengths 
are important for constraining the stellar mass and age, 
but the reader should remember that the IRAC points 
only correspond to a subset of the objects used for the 
r'i'z'YJHKs stacks. 

The resulting median stacked images are shown in Fig- 
ure[ni There are significant detections at all filters except 
for r' band. Photometry of the ground-based data was 
performed to obtain 3 arcsec aperture magnitudes or lim- 
its. These are equivalent to total magnitudes, within the 
uncertainties. For the IRAC data which has a broader 
PSF, 3.8 arcsec apertures were used and a PSF correction 
to total magnitudes applied. Smoothing of the images 
showed that for all the optical filters, there is a negative 
"hole" in the background covering the central five arcsec- 
onds. This is most apparent for r' and z' in Figure|6l This 
is showing us that our z — 6 galaxy sample is incomplete. 
The CFHTLS Deep r' and i' band data is so deep that it 
is close to the confusion limit and we are missing z = 6 
galaxies that have foreground galaxies contaminating the 
photometry aperture in r' and i' . Another consequence 
of this is that we are unlikely to have strongly gravita- 
tional lensed galaxies in our sample due to the lensing 
galaxy contaminating the source galaxy aperture. Pho- 
tometric measurements were performed carefully to en- 
sure that the appropriate background level was set. The 
measured magnitudes of the stacks are given below the 
images in Figure [6l The 2a limit on the r' band magni- 
tude (determined from the noise in the background pixels 
because this region has lower variance than the rest of 
the field which was populated by galaxies in the input 
images) gives r' > 29.8. This is almost 5 magnitudes 
fainter than the z' band magnitude and provides com- 
pelling evidence that these galaxies are truly at z « 6 
rather than lower redshift galaxies or dwarf stars. The 
i' — z' and z' — J colors of the stack are also entirely 
consistent with a typical z = 5.9 galaxy (as discussed in 
Section [^31 and plotted on Figure [5]). 

Figure [7] plots the stack fluxes as a function of wave- 
length. The z' flux has been reduced by a factor of 1.1 
to account for the typical Lya emission line contribu- 
tion. It is apparent that the YJHKs fluxes are described 
by a power-law with index (deflned as f\ oc X'^) redder 
than /? = — 2 (which would be flat on this plot which 
uses /^). A power-law flt to the YJHKs fluxes gives 
(3 — —1.44 ± 0.10, where the la uncertainty comes from 
1000 bootstrap resample trials. To ensure that this re- 
sult is not an artifact of the stacking method, we have 
also fitted the YJHKs fluxes of the 23 LBGs individ- 
ually. Excluding two extreme outliers which have poor 
fits, it is found that the mean /3 — —1.38 and the median 
/3 — —1.02. The standard error on the mean is ±0.20. 

The value of /? found is significantly redder than the 
typical values previously observed for the most luminous 
z « 6 LBGs in deep Hubble Space Telescope surveys of 
(3 = -2.04±0.17 (Bouwens et al. 2011), ^ = -2.10±0.16 
(Dunlop et al. 2012), ^ = -2.04 ± 0.17 (Wilkins et 
al. 2011) and /? = -2Mtoll (Finkelstein et al. 2011). 
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Figure 6. Median stacked images in the r'i' z'YJHKs and IRAC 3.6/im and 4.5/im filters at tlie positions of the LBGs used to construct 
the stacks. There are 23 galaxies in the optical (CFHTLS) and YJHK (VIDEO and UltraVISTA) stacks and 15 in the IRAC bands 
(S-COSMOS). Each image covers 10" X 10". The images are oriented with north up and east to the left. Negative holes in the central few 
arcsec of the images (particularly apparent in r' and i') are due to incompleteness of our LBG sample due to foreground contamination. 
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Figure 7. Photometry from the stacked images (blue points with 
error bars) plotted with stellar evolution synthesis model (black 
line). The galaxy model is fixed at z = 5.9 and described in the 
text. The z' band flux has been reduced by a factor of 1.1 to 
account for the typical Ly« emission, which is not included in the 
models. 

Wilkins et al. also used a bin containing only the 4 most 
luminous LBGs with —21.5 < A/1500 < —20.5 where they 
found /3 — —1.58 ± 0.09, which is similar to our result. 
Finkelstein et al. showed that although at z « 6 there 
is only a weak correlation between /3 and UV luminosity, 
there is a stronger correlation between /? and stellar mass. 
Whilst this is not surprising, because in a UV flux-limited 
sample, a redder galaxy model will require a higher stel- 
lar mass, the typical value of /3 for their most massive bin 
(10^ to lO^^Mo) is -1.73 and the typical stellar mass of 
our sample is lO^^M© (see below). The simplest expla- 
nation for dust in luminous galaxies comes from com- 
bining the metallicity-dust, mass-metallicity and mass- 
luminosity correlations. Therefore our typical /? = —1.4 
for z « 6 luminous LBGs is not at odds with the work of 
Finkelstein et al. Our findings suggest that a substantial 
number of these galaxies have significant dust reddening. 

Galaxy models were fitted to the stacked photome- 
try using the photometric redshift method of McLure 
et al. (2011). The fitted models used Bruzual & Chariot 
(2003) stellar populations with the Chabrier IMF and 
the Calzetti et al. (2000) dust attenuation law. Vari- 
ous star formation histories were fitted, but none were 
preferred at a high significance. Therefore we quote the 
constant star formation rate results, since these most- 
luminous LBGs are unlikely to be observed in a state of 



rapidly declining or increasing star formation rate. Fig- 
ure [7] shows the best-fit model and its parameters. The 
best fit redshift was found to be z = 5.95 with a 2a 
range of 5.87 < z < 5.97. The best-fit metallicity is so- 
lar. The best-fit stellar mass, M^, is 1.1 x IO^^Mq with 
a 2cr range of 4 x 10^ < < 1.9 x IO^^Mq. The age 
and star formation rate are obviously degenerate with the 
star formation history and dust reddening, so no strong 
constraints could be placed on those parameters. As was 
found in the analysis of (3, the UV spectral slope is fairly 
red and this requires significant dust reddening. The best 
fit has dust extinction of Ay = 0.75 and the 2a range is 
0.48 < Ay < 1.48. 

6. GALAXY LUMINOSITY FUNCTION 
6.1. Completeness 

There are a number of factors that affect the com- 
pleteness of the sample, or equivalently, the effective vol- 
ume of the survey. Firstly, the sample does not contain 
every object brighter than a certain z' magnitude limit 
due to incompleteness in the source detection algorithm 
close to the limit. In addition, blending with brighter 
objects becomes an issue in ground-based data at these 
faint magnitudes. As was shown in the stacks of Section 
[HI there is evidence for incompleteness of the sample due 
to foreground contamination at i' band which prevents 
some true high-redshift galaxies from having i' — z' > 2 
in their aperture magnitudes. 

In order to determine the effective sky area surveyed, 
we have carried out an analysis of insertion and recovery 
of simulated sources into the data. As found in Sec- 
tion m about 50% of the z = 6 LBGs are spatially re- 
solved, whereas 50% are consistent with point sources. 
The simulated sources have a similar distribution of ob- 
served FWHM to the LBGs. The selection criteria for 
recovery are identical to those of the automated candi- 
date selection. In addition, we check whether there is a 
contaminating object at i' band which would cause the 
observed i' — z' color to fall below the threshold. Figure 
[8] shows the effective sky area surveyed in all four Deep 
fields as a function of the input magnitude of the arti- 
ficial galaxies. The dashed line shows the effective area 
if one does not consider the foreground contamination 
of the i' — z' color. The effective area declines slowly 
from z' — 23 to z' = 24.8 and then falls more rapidly 
to z' = 25.5. Note that the effective area is still consid- 
erable for input magnitudes 25.3 < z' < 25.5, so some 
galaxies fainter than the nominal magnitude limit will be 
in the sample due to photometric scatter. 

The next factors to consider for completeness are the 
effects of the i' — z' > 2 color criterion and the conver- 
sion from z' magnitude to absolute magnitude. These 
are assessed by considering the range of properties ex- 



10 



Willott et al. 



4 r ' 1 ' 1 ' 1 ' r 




23.0 23.5 24.0 24.5 25.0 25.5 26.0 
z magnitude 

Figure 8. Effective survey area of tiie LBG sample as a function 
of magnitude (solid line) . Note that these magnitudes are the true 
magnitudes of the simulated galaxies before photometric noise is 
added, so some galaxies fainter than the magnitude limit (z' = 
25.3; dotted line) arc included in the sample. The dashed line 
shows what the effective area would be without incompleteness 
due to foreground contamination of the i' band magnitude and 
hence color. 
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Redshift z 

Figure 9. Completeness due to the color criterion of i' ~ z' > 2 
as a function of redshift. This curve was determined for a popula- 
tion of galaxies with Lya equivalent width, UV continuum slope, 
IGM absorption and photometric errors as described in the text. 
Because all objects with i' — z' > 2 have been identified as ci- 
ther galaxies or brown dwarfs there is no high redshift cutoff in 
completeness corresponding to high z' — J colors. 

pected for high-redshift galaxies, including Lya emission 
line strength, UV continuum spectral slope and IGM ab- 
sorption of the UV continuum. As in Section [3] we use 
the z — 'i LBG composite of Shapley et al. (2003) as the 
basis for the galaxy template. Absorption shortward of 
Lya due to foreground neutral hydrogen is modeled as in 
Willott et al. (2010) using the mean and scatter derived 
by Songaila (2004). 
A range of UV spectral slopes is implemented, as re- 



quired by observations. At z w 6 and high luminosity 
the reported typical value in the literature is (3 = —2 
(Bouwens et al. 2011; Finkelstein et al. 2011). Our 
stacking and individual LBG fitting results showed that 
for even more luminous z « 6 LBGs, the typical /3 = 
— 1.4. Combining our results with other work, we as- 
sume a typical value for the population of /? = — 1.8 and 
a Gaussian distribution with Icr scatter of 0.5. We have 
checked and using a mean of /3 = — 1.5 or /? = — 2 makes 
no difference to our results. 

Several spectroscopic studies have investigated the dis- 
tribution of Lya equivalent widths at high redshifts. We 
combine the works of Stark et al. (2011), Curtis-Lake et 
al. (2011) and our own work in Section|3]to derive a Lya 
rest-frame equivalent width probability distribution for 
luminous z w 6 LBGs of P(WLya) oc exp(— WLyQ/25). 
This is similar to the exponential parameterization of Di- 
jkstra & Wyithe (2012), also based on the data of Stark 
et al. 

A sample of 500 simulated galaxies were generated with 
properties randomly drawn from these distributions and 
their j'z'J colors and absolute magnitudes, M1350, de- 
termined if located at all redshifts between z = 5.5 and 
z = 6.7. Photometric scatter in the colors is also in- 
cluded. The median i' — z' and z' — J color as a function 
of redshift is plotted in Figure [5] This curve runs very 
close to the stacked values of i' — z' and z' — J, suggest- 
ing the models are indeed a good representation of the 
data. At redshifts 5.6 < z < 6, some fraction of galaxies 
are too blue in i' — z' to be included in the sample. In 
particular, we find that galaxies with strong Lya lines at 
z < 5.9 are much less likely to be included in our sam- 
ple than those with weak Lya lines. This could be the 
reason why we find a relatively low Lya fraction for the 
galaxies with best-fit redshifts z w 5.8 (c.f. Curtis-Lake 
et al. 2011). At z > 5.9, where this bias does not exist, 
the fraction with strong Lya in our sample is quite high 
(two out of three). Figure [S] shows the completeness as a 
function of redshift due to the color selection criterion. 

6.2. Magnitude distributions 

Before using these data to derive the z = 6 galaxy 
luminosity function, we consider simply the observed z' 
band magnitude distributions of the galaxies and brown 
dwarfs. Figure [10] shows these two distributions. The 
dwarf magnitudes extend to z' = 22.8 and have a fairly 
flat distribution with a peak at z « 25.0. This flat dis- 
tribution is a consequence of galactic structure and the 
survey completeness as a function of magnitude. At mag- 
nitude z' — 25, mid-L and mid-T dwarfs are 500 and 200 
pc away, respectively. The CFHTLS fields are at high 
galactic latitude and hence the space density of brown 
dwarfs declines beyond the disk. The galaxy distribution 
is quite different and much steeper. There are no galaxies 
brighter than z' = 24.4 and almost 70% of the galaxies 
lie in the small magnitude range of 25 < z' < 25.3. The 
steepness of this distribution, despite the declining sur- 
vey completeness at faint magnitudes, is due to the very 
steep bright end of the galaxy luminosity function. 

The gray histogram in Figure [10] shows the expected 
magnitude distribution for z = 6 LBGs in our survey 
based on the evolving luminosity function in Section 5.3 
of Bouwens et al. (2008). At 3.5 < z < 6.5 this lu- 
minosity function is constrained by the LBG luminosity 
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Figure 10. Magnitude histograms for Lyman break galaxy eandi- 
dates and brown dwarfs seleeted by colors i' — z' > 2 and 2' < 25.3. 
The brown dwarf magnitude distribution (blue) has a fairly shallow 
slope due to the fact that the distance to L dwarfs is well beyond 
the disk scale length. The galaxy magnitude distribution (red) 
has a very steep slope. The gray shaded histogram shows the ex- 
pected LBG counts for our survey based on the known luminosity 
function which is well-constrained at fainter magnitudes (Bouwens 
et al. 2008). The dashed line shows the magnitude limit of our 
samples. 

functions determined in Bouwens et al (2007) based on 
ACS observations in the GOODS, HUDF and HUDF- 
Parallel fields. The expected counts take into account 
the selection criteria, completeness and photometric un- 
certainty of our sample. It is clear from Figure [10] that 
both the shape and overall normalization of the expected 
counts are close to those observed in our sample. This 
is remarkable given that our survey volume is 40 times 
greater than that used by Bouwens et al. (2007) and 
hence their data contained very few sources as bright as 
ours. The expected magnitude distribution predicts 46 
LBGs in our sample, compared with 40 observed. The 
number expected at z' < 25 is 13, identical to that ob- 
served. 

6.3. Luminosity function derivation 

Because our LBG sample covers only a limited range 
of apparent and absolute magnitudes, we cannot use it to 
determine the full galaxy luminosity function at z = 6. 
The luminosity function at the break and at fainter mag- 
nitudes has already been well-studied from deep Hubble 
Space Telescope surveys over small sky areas (Bouwens 
et al. 2007). The main contribution of our study is to 
determine the space density of very rare, highly luminous 
LBGs. 

Due to the fact that most CFHTLS LBGs have un- 
known redshifts (within the Lyman break redshift se- 
lection range) and unknown /c-corrections due to Lya 
emission line contribution and spectral slope, we can- 
not map a particular apparent magnitude onto an ab- 
solute magnitude and redshift. Therefore we cannot 
carry out standard luminosity function derivation meth- 
ods where each observed source corresponds to a point 
on the redshift, luminosity plane or by counting the 
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number of galaxies in luminosity bins. Instead we use 
the stepwise maximum-likelihood method of Efstathiou 
et al. (1988) where the luminosity function is charac- 
terized by values in a number of absolute magnitude 
bins. We choose to use 5 regularly spaced nodes at 
Afi35o = [-22.5,-22,-21.5,-21,-20.5] since this cov- 
ers the full extent of the absolute magnitude range of 
our detected LBGs and ensures that extrapolation of the 
luminosity function beyond this range will not bias our 
results. 

The comparison of the model with the data is per- 
formed in observed magnitude space because this is the 
only way of accounting for the selection effects and com- 
pleteness discussed in Section 16.11 and the non-unique 
mapping of apparent to absolute magnitude. For a model 
luminosity function (defined as power-laws connecting 
and extending beyond the five nodes), the completeness 
information and photometric noise are used to generate 
a model observed z' magnitude distribution, using the 
same method as in Section 16.21 These model magni- 
tudes are binned in Sz' = 0.1 magnitude bins, rimod, and 
compared to the observed z' magnitude histogram, rtobs, 
(Figure [T0| . The data are Poissonian and the likelihood, 
C, of observing the data given the model over all N mag- 
nitude bins is given by 

N -T!„,-,H i "oba.i 

c = l[- ^f^. (1) 

2—1 ' 

The galaxy space densities at the five nodes are the five 
free parameters which are adjusted using an amoeba al- 
gorithm (Press et al. 1992) to determine the maximum 
likelihood. The 68% range on the best fit values are 
determined via bootstrapping by 400 trials of Poisson 
perturbation of the nobs.i values. 

The results are given in Table 3. Figure [TT] plots the 
data and compares with previous results from the lit- 
erature. Our best-fit values match well the Schechter 
parameterizations of Bouwens et al. (2007, 2008) and 
McLure et al. (2009) and agree with previous binned 
values determined by Bouwens et al. (2007) and McLure 
et al. (2009). Our two most luminous data points show 
the steep decline in space density towards high luminos- 
ity required by our data. The 68% uncertainties on our 
data points are fairly large due to the small sample size 
and the fact that sources can be redistributed to neigh- 
boring nodes in different trials. The total volume of our 
survey is w 10^ Mpc. Therefore we are not truly measur- 
ing a space density of < 10-''Mpc-^ at M1350 = —22.5. 
The low value of this node is showing that there must 
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Figure 11. Galaxy luminosity function at 2 6. The maximum 
likelihood node points determined in this paper from CFHTLS are 
shown by filled black circles, whilst binned points from the liter- 
ature are shown with open black symbols (GOODS+HUDF from 
Bouwens et al. 2007 and SXDS/UDS from McLure et al. 2009). 
Schechter function curves are plotted from the works of Bouwens et 
al. (2007) (red), Bouwens et al. (2008) (green) and McLure et al. 
(2009) (blue). The CFHTLS data at the bright end most closely 
match the McLure et al. derivation. 

be a sharp decline brighter than M1350 = —22 or else we 
would observe more bright LBGs. One of our spectro- 
scopically confirmed galaxies, WMH5, has M — —22.65, 
but most of the rest have absolute magnitudes close to 
-^1350 = ~22, in agreement with this sharp decline in 
number density. 

6.4. Field-to-field variance 

Part of the robustness of our results comes from the 
fact the CFHTLS Deep is spread over four separate 1 
square degree fields. We now consider the variance in 
z « 6 LBG counts across the four fields. We identified 
8, 15, 8 and 9 LBGs in Dl, D2, D3 and D4, respectively. 
The two fields with the least certain identifications due 
to shallower J-band (D3 and D4) do not have an unusual 
number of counts, providing confidence in the LBG clas- 
sification in these fields. The field with the largest de- 
viation is D2 (COSMOS) which, with 15 LBGs, is 1.6a 
beyond the mean, assuming Poisson variance only. An- 
other source of variance is the large-scale distribution of 
matter, so called cosmic variance. This could be im- 
portant if the most luminous LBGs are hosted by rare, 
massive dark matter halos. We use the cosmic variance 
calculator of Trenti & Stiavelli (2008) assuming a duty 
cycle of 0.5 for the halos. This shows that the cosmic 
variance contribution to the total variance in each field 
is expected to be only half the Poisson variance contri- 
bution. Hence including cosmic variance the D2 field is 
1.4cr beyond the mean. 

As well as chance, another factor that could be con- 
tributing to the higher counts in D2 is that the CFHTLS 
data is a little shallower in that field. With shallower 
data, there are three possible effects that alter the ex- 
pected number of candidates found: (i) the larger mag- 
nitude uncertainties cause more objects to scatter in from 



faintward of the z' = 25.3 magnitude limit; (ii) the larger 
magnitude uncertainties cause more objects to scatter in 
from blueward of the i' — z' = 2 color limit; (iii) the de- 
tection completeness will be lower at the faint end. All 
of these effects are included for the sample as a whole in 
Section r6.11 but we do not analyze them on a field-by-field 
basis. Certainly, effect (i) is not significant because com- 
parison of the CFHTLS z' and Subaru z' magnitudes in 
D2 shows that only two of the 15 have Subaru z' > 25.3. 
A similar fraction is found for Dl using VIDEO Z mag- 
nitudes and is expected based on Figure [5) Therefore we 
ascribe the higher number of LBGs in D2 being mostly 
due to chance and the total of 40 in the sample is unbi- 
ased and fully accounted for by our selection function. 

6.5. Discussion 

Our results show that there is a sharp decline in the z « 
6 galaxy space density for galaxies brighter than A/1350 = 
—22, consistent with the exponential function of Bouwens 
et al. (2007; 2008). The steepness of this function is 
much steeper than that of the dark matter halo mass 
function for the expected host halo masses, Mh, of these 
luminous LBGs (lO^^ <^ Mr < 10'^^ Mq] Lee et al. 2009; 
Trenti et al. 2010). This implies suppression of star 
formation in the most massive halos at high redshift. The 
galaxy luminosity and dark matter halo mass functions 
can be related using the conditional luminosity function 
(CLF) method (Yang et al. 2003). At low redshifts, the 
bright end of the galaxy luminosity function is consistent 
with the relationship L - A/j^^s ^y^^ig ^ Qstriker 2006). 
The physical explanation for this is likely feedback from 
an AGN and/or inefhcient gas cooling in high mass halos 
(Benson et al. 2003). 

Because our best-fit luminosity function is so similar 
to that of Bouwens et al. (2007), we can adopt the z — 
6 CLF derivation of Trenti et al. (2010) which was fit 
to this function. Trenti et al. (2010) showed that by 
requiring only recently formed (200 Myr) halos to host 
LBGs, the duty cycle for high mass halos must be almost 
unity, unlike the values of « 20% previously estimated 
for typical LBGs at this epoch (Stark et al. 2007; Lee 
et al. 2009). Under this assumption that the duty cycle 
does not vary much over the luminosity range of interest, 
Trenti et al. (2010) showed that the high luminosity end 
of the CLF follows a relationship of L Af^^. This is 
somewhat steeper than is found at low redshift, but still 
indicates that luminosity is a slowly varying function of 
halo mass. 

Models with star formation efhciency and duty cycle 
(or star formation timescale) which are independent of 
halo mass do not predict such a steep decline in the lu- 
minosity function at the brightest magnitudes (Stark et 
al. 2007; Munoz & Loeb 2011). These works were able 
to fit the z = 6 luminosity function of Bouwens et al. 
(2007) because those data did not provide evidence of 
the sharp exponential decline. Our results show that the 
same process that limits star formation in high mass ha- 
los at low redshift is also acting at a time just one billion 
years after the Big Bang. 

7. CONCLUSIONS 

We have performed the largest area survey for lumi- 
nous LBGs at z « 6. These galaxies have been used to 
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study their physical sizes and spectral energy distribu- 
tions. The large sky area has enabled the most robust 
results on the space density decline at the bright end of 
the luminosity function. Below we list the specific con- 
clusions drawn from this work. 

• Spectroscopy of seven galaxies revealed redshifts 
for five of them. Three of these have Lya emission 
lines, one of which is extremely weak with H^Lya = 
4A. All five galaxies show continuum redward of 
Lya which enables redshift determination from the 
break location. 

• About half of the LBGs are resolved in the ground- 
based CFHT imaging observations. This is con- 
sistent with a typical intrinsic half-light radius for 
luminous 2: « 6 galaxies of 2 kpc, higher than the 
typical 1 kpc in GOODS. High resolution imaging 
of a galaxy with M1350 = —22.27 shows that the 
main component is actually quite compact (half- 
light radius = 0.5 kpc) and the extended nature 
from the ground is due to an interacting compo- 
nent with 20% of the flux. A larger sample with 
high resolution imaging is necessary to determine 
if there is a relationship between interaction fre- 
quency and star-formation rate. 

• The properties of the stacked photometry have 
been investigated. These data show a flux decre- 
ment in the observed-frame optical of nearly 100 
due to the Lyman break. The YJHK photometry 
are fit by a power-law of slope /? — —1.44 ± 0.10, 
much redder than for less luminous galaxies at 
this epoch. The stacked SED is well fit by a 
z = 5.95 constant star formation model with mass 
^ IO^^Mq. Significant dust reddening is required 
{Ay = 0.75) showing that dust is more prevalent 
in the most massive systems, again possibly due to 
these being merger-induced starbursts. 

• The magnitude distributions of our LEG sample 
and a i' — z' color- matched sample of L/T dwarfs 
are considered. Whilst the dwarfs have a fairly flat 
distribution with a peak at z' ~ 25, the LBGs have 
a distribution that rises sharply with magnitude. 
Our z « 6 luminosity function derivation provides 
clear evidence for a sharp decline in space density 
brighter than M = -22. 

• The decline in space density at high luminosity is 
consistent with a relationship between galaxy lu- 
minosity and dark matter halo mass of L ~ Mf^^. 
This shows that the processes limiting star forma- 
tion in high mass halos at low redshift are also op- 
erating effectively at a time just 1 billion years after 
the Big Bang. 
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